Intel Realistic Sound Experience (3D RSX) Handout

IAL Home | Technology Home | Additional Documentation

Intel Realistic Sound Experience (3D RSX)

Realistic Sound for the Internet

A Demonstration of Realistic Sound
Integrated into VRML Worlds

Background -- The need for realistic audio in VRML

While virtual environments are primarily defined in terms of graphics, sound is also an essential element of the immersion process. Sound can complement the use of graphic objects in three key areas:

Spaciousness -- While our eyes are focused ahead of us in a relatively narrow field of view, sound can simultaneously feed us information about our full surrounding environment. This continuous, low resolution information stream provides us with the ability to guide our high-resolution mechanism (i.e. our eyes) to points of interest.
Quality -- Through sound it is possible to virtually recreate the sensory experience of a physical environment. The state of the art for audio in this area is way beyond the capability of any other media type and is an essential element of the immersion illusion.
Information Content -- As a completely orthogonal sensory experience to our visual input, sound provides a high bandwidth information channel that directly influences a user's perception of a virtual environment.

Most of the existing standards for 3D worlds only provide lip-service to audio (pun - intended). While technologists generally agree that audio benefits a virtual environment, there seem to be two fundamental barriers to its wide adoption:

A standard format for compactly describing an audio object for a 3D world in a platform independent manner.
A set of 3D audio rendering libraries for the volume platform.

VRML, as the de-facto standard for defining platform independent virtual worlds, is in a key position to overcome this first barrier. The inclusion of a standard audio node to the VRML specification will allow scene designers to take advantage of audio in a platform independent manner. The concept demonstrations shown by Intel are designed to address the second barrier to the use of realistic audio in VRML scenes -- a set of 3D audio libraries for the volume platform. These demonstrations show that fully immersive sound can run efficiently and effectively on the volume platform. Realistic sound, through Intel's RSX (Realistic Sound Experience) is realizable on volume platforms The quality of the demonstrations follows the model of graphics rendering engines, which is based on the concept of scalability: On very low end platforms, the quality is recognizable, but not outstanding. On the current volume platform, very high quality audio is realizable, but at the cost of some CPU overhead. Finally, on high end platforms with hardware accelerators, this same high quality can be achieved, while freeing up the CPU for other computational tasks.

A range of localization algorithms have been implemented to compare the tradeoffs between localization quality and CPU speed for host-based audio processing. While a "modified panning" algorithm has proven to be cost effective, a high quality "HRTF" based algorithm, running multiple streams, is realizable on the host platform for a fraction of the total CPU cost. In addition to localization, reverberation and Doppler shift are effective in rendering realism in the audio portion of a virtual environment.

VRML'95 - Concept Demonstration

As part of the VRML'95 conference, Intel is demonstrating realistic audio in the context of two different applications. The first application vividly demonstrates how realistic audio can enhance a static world. The "SoundTracker" application allows a user to take a standard bitmap, representing a 2D world, and add spatial audio to it. The image is transformed from a picture to an entire spatial environment that begs exploration. In addition to showing how sound can be added to the current class of VRML content, the SoundTracker application validates the use of the audio node that we are proposing for inclusion in the next VRML specification revision.

The second application for is a fully interactive 3D scene which not only allows the viewer to move, but also contains dynamic graphical and audio elements. This application demonstrates multiple simultaneous audio sources in 3D space interacting with a moving "listener" object. The complexity of this demonstration illustrates that high quality, interactive audio localization is practical on the host processor at a CPU cost that leaves plenty of overhead for graphics and game logic.

Both demonstrations have been prepared using a set of COM libraries that take advantage of the Pentium® Processor for audio localization and reverberation in real time. The sound libraries currently run on Windows^* 95 and use Microsoft's DirectX^* as the audio output interface. Intel has developed these libraries for the purpose of evaluating realistic sound in a variety of application domains.